{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X0doHtrEl66V"
      },
      "source": [
        "##### Copyright 2025 Google LLC."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 30,
      "metadata": {
        "cellView": "form",
        "id": "2K6KP3hUl7_l"
      },
      "outputs": [],
      "source": [
        "# @title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sP8PQnz1QrcF"
      },
      "source": [
        "# Gemini API: Basic evaluation"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bxGr_x3MRA0z"
      },
      "source": [
        "<a target=\"_blank\" href=\"https://colab.research.google.com/github/google-gemini/cookbook/blob/main/examples/prompting/Basic_Evaluation.ipynb\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" height=30/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ysy--KfNRrCq"
      },
      "source": [
        "Gemini API's Python SDK can be used for various forms of evaluation, including:\n",
        "- Providing feedback based on selected criteria\n",
        "- Comparing multiple texts\n",
        "- Assigning grades or confidence scores\n",
        "- Identifying weak areas\n",
        "\n",
        "Below is an example of using the LLM to enhance text quality through feedback and grading."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "metadata": {
        "id": "Ne-3gnXqR0hI"
      },
      "outputs": [],
      "source": [
        "%pip install -U -q \"google-genai>=1.7.0\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 32,
      "metadata": {
        "id": "EconMHePQHGw"
      },
      "outputs": [],
      "source": [
        "from google import genai\n",
        "from google.genai import types\n",
        "from IPython.display import Markdown"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eomJzCa6lb90"
      },
      "source": [
        "## Configure your API key\n",
        "\n",
        "To run the following cell, your API key must be stored it in a Colab Secret named `GOOGLE_API_KEY`. If you don't already have an API key, or you're not sure how to create a Colab Secret, see [Authentication](https://github.com/google-gemini/cookbook/blob/main/quickstarts/Authentication.ipynb) for an example."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 33,
      "metadata": {
        "id": "v-JZzORUpVR2"
      },
      "outputs": [],
      "source": [
        "from google.colab import userdata\n",
        "GOOGLE_API_KEY=userdata.get('GOOGLE_API_KEY')\n",
        "\n",
        "client = genai.Client(api_key=GOOGLE_API_KEY)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "L-Wt23A_uzFZ"
      },
      "source": [
        "## Example\n",
        "\n",
        "Start by defining some system instructions for this problem. For demonstration purposes, the use case involves prompting the model to write an essay with some mistakes. Remember that for generation tasks like writing an essay, you can change the temperature of the model to get more creative answers."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 34,
      "metadata": {
        "id": "zNw1Hele0Bne"
      },
      "outputs": [
        {
          "data": {
            "text/markdown": "Reading is like, a really good thing. It’s beneficial, you know? Like, a lot. That's what I'm gonna talk about. So, like, my thesis is that reading is really important for a bunch of reasons that I will now tell you.\n\nFirst off, reading makes you smart. I read this somewhere, I forget where. Probably on the internet. But, like, think about it. When you read, you learn new words. And new words are good because you can use them to impress people. The more words you know, the smarter you seem. Its like a secret code to being smart, I think. My English teacher is always saying, “use a thesaurus!” So, like, I do. Sometimes. Its useful. Plus, reading exposes you to new ideas, and new ideas are good for…stuff.\n\nSecondly, reading books is way better than watching TV. TV rots your brain. Everyone knows that. I mean, have you ever met someone who watches TV all day and is, like, a genius? No, right? Exactly. But books, books are different. Books make you think, even if the stuff you read doesn't make much sense. I read this book last week, something with like, a red cover, and it didn't make any sense. But I know it was making me smarter, because I kept having to go back and try to understand it. Therefore, books good, TV bad.\n\nFinally, reading can help you with writing. The more you read, the more you pick up on how other people write, their, like, style and stuff. So when it comes time for you to write something, like this essay even, it will be easier. It's like you’re absorbing their talent through your eyeballs. I think it's osmosis or something. And also, it's because you learn all these new words, that I mentioned before. I always learn a whole bunch of new words whenever I am reading. Its just natural, you know.\n\nIn conclusion, reading is really, really good. It makes you smart, its better than watching TV, and it helps you with writing. So everyone should read more. Seriously, go read a book. I did, and look at me, writing this essay! Like, I am super benefiting from reading right now. You should to. Also, I hope I get a good grade on this essay.\n",
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "execution_count": 34,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "student_system_prompt = \"\"\"\n",
        "    You're a college student. Your job is to write an essay riddled with common mistakes and a few major ones.\n",
        "    The essay should have mistakes regarding clarity, grammar, argumentation, and vocabulary.\n",
        "    Ensure your essay includes a clear thesis statement. You should write only an essay, so do not include any notes.\n",
        "\"\"\"\n",
        "\n",
        "MODEL_ID=\"gemini-2.5-flash\" # @param [\"gemini-2.5-flash-lite\",\"gemini-2.5-flash\",\"gemini-2.5-pro\",\"gemini-3-pro-preview\"] {\"allow-input\":true, isTemplate: true}\n",
        "\n",
        "student_response = client.models.generate_content(\n",
        "    model=MODEL_ID,\n",
        "    contents='Write an essay about the benefits of reading.',\n",
        "    config=types.GenerateContentConfig(\n",
        "        system_instruction=student_system_prompt\n",
        "    ),\n",
        ")\n",
        "\n",
        "essay = student_response.text\n",
        "Markdown(essay)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 35,
      "metadata": {
        "id": "mzp8ZFXf0-OT"
      },
      "outputs": [
        {
          "data": {
            "text/markdown": "Okay, here's a breakdown of your essay, along with a corrected version.\n\n**Evaluation:**\n\n*   **Thesis Statement (2/5):** The thesis is present but lacks sophistication and clarity. It's too conversational and doesn't precisely outline the specific arguments.\n*   **Clarity and Precision of Language (2/5):** The language is overly informal, repetitive (\"like,\" \"you know\"), and lacks precision. The writing could benefit from more specific examples and a more academic tone.\n*   **Grammar and Punctuation (3/5):** There are noticeable errors in grammar and punctuation, including incorrect use of \"its\" vs. \"it's,\" run-on sentences, and comma splices.\n*   **Argumentation (2/5):** The arguments are simplistic and lack strong evidence. The connections between reading and intelligence, writing ability, etc., are not fully developed or supported with credible sources.\n\n**Corrected Essay:**\n\nReading offers multifaceted benefits that extend beyond mere entertainment. Its importance is underscored by its capacity to enhance cognitive abilities, provide a superior alternative to passive screen time, and cultivate effective writing skills.\n\nFirstly, reading significantly contributes to intellectual development. Exposure to a wide range of texts expands vocabulary and enhances comprehension. As one encounters new words and concepts within various contexts, they are integrated into one's cognitive framework, facilitating more nuanced and articulate communication. As such, reading becomes a powerful tool for self-improvement and intellectual growth. Furthermore, reading exposes individuals to diverse perspectives and ideas, which stimulates critical thinking and broadens one's understanding of the world.\n\nSecondly, engaging with literature offers a more enriching experience compared to passive forms of media consumption like television. While television often presents information in a simplified and visually driven format, reading requires active engagement and interpretation. This active engagement fosters concentration, analytical skills, and imagination, leading to deeper cognitive processing. Moreover, the ability to select books allows for a tailored and purposeful learning experience, unlike the often random and unfiltered nature of television programming.\n\nFinally, reading plays a crucial role in the development of writing proficiency. By immersing themselves in the works of skilled authors, readers subconsciously absorb stylistic elements, narrative structures, and effective rhetorical techniques. This implicit learning process enhances one's ability to articulate thoughts clearly, construct compelling arguments, and employ language effectively. Furthermore, the increased vocabulary acquired through reading directly translates into more precise and varied writing.\n\nIn conclusion, reading offers a wealth of cognitive, intellectual, and creative advantages. Its capacity to enhance intellect, surpass the limitations of passive media, and foster effective writing skills underscores its importance in personal and academic development. Therefore, individuals are encouraged to embrace reading as a means of lifelong learning and intellectual enrichment.\n\n**Explanation of Changes:**\n\n*   **Thesis Statement:** The original thesis was very informal. The corrected version is more formal and specific, clearly stating the three main benefits.\n*   **Language:** All instances of \"like,\" \"you know,\" and other informal expressions were removed. The language was made more precise and academic.\n*   **Grammar and Punctuation:** Grammatical errors were corrected (e.g., \"its\" changed to \"it's\"). Run-on sentences were split, and commas were added or removed for clarity.\n*   **Argumentation:** The arguments were strengthened by providing more specific explanations of how reading leads to intelligence, better writing, etc. The vague references to \"reading it somewhere\" were removed. While still lacking direct citations, the corrected version presents more logical reasoning.\n*   **Structure:** The essay was structured with clear topic sentences for each paragraph and a more formal concluding paragraph.\n*   **Tone:** The overall tone was shifted from conversational to academic.\n\n",
            "text/plain": [
              "<IPython.core.display.Markdown object>"
            ]
          },
          "execution_count": 35,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "teacher_system_prompt = \"\"\"\n",
        "    As a teacher, you are tasked with grading students' essays.\n",
        "    Please follow these instructions for evaluation:\n",
        "\n",
        "    1. Evaluate the essay on a scale of 1-5 based on the following criteria:\n",
        "       - Thesis statement,\n",
        "       - Clarity and precision of language,\n",
        "       - Grammar and punctuation,\n",
        "       - Argumentation\n",
        "\n",
        "    2. Write a corrected version of the essay, addressing any identified issues\n",
        "       in the original submission. Point out what changes were made.\n",
        "\"\"\"\n",
        "teacher_response = client.models.generate_content(\n",
        "    model=MODEL_ID,\n",
        "    contents=essay,\n",
        "    config=types.GenerateContentConfig(\n",
        "        system_instruction=teacher_system_prompt\n",
        "    ),\n",
        ")\n",
        "\n",
        "\n",
        "Markdown(teacher_response.text)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4Hbzc0EVDHxS"
      },
      "source": [
        "## Next steps\n",
        "\n",
        "Be sure to explore other examples of prompting in the repository. Try writing your own prompts for evaluating texts."
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "Basic_Evaluation.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
